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The effectiveness of Web based instruction: A case study 



With the advent of the World Wide Web in 1993, more and more people are finding that 
they can shop, chat, search for information, send messages, etc. without leaving their homes. Not 
to be left out of the information revolution, colleges and universities have begun offering courses 
via the Internet. 

These web-based courses follow a long history of experiments with distance learning. A 
more traditional format for distance learning is the “telecourse,” a video-based course. Today’s 
most innovative “telecourses” tend to be live and interactive (Biner et.al. 1996, Biner et.al. 1997, 
Falck et.al. 1997). A live, interactive “telecourse,” however, requires that students congregate in 
one or more designated locales at a specified time. 

Web-based courses allow students to complete the course work on a schedule they set for 
themselves. In this respect, the web-based course is most like the type of correspondence school 
course using printed packages of lectures and assignments. But web-based courses can 
incorporate some of the elements of a “telecourse” as well - such as interactivity and audio/video 
elements (though the latter are rare due to hardware constraints). 

A consortium of community colleges in the United States created the Community College 
Distance Learning Network. This network is distributed nationally and includes courses offered 
via the Internet, video-based “telecourses,” and combinations of the two. (“Community 
Colleges. . .” 1998). The newly formed Florida Gulf Coast University was designed and built as a 
testing ground for Internet-based distance Learning (‘TGCU Faculty...” 1998). And Jones 
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International University, which specializes in selling Internet-based courses for profit, is the first 
Internet-only school to be accredited to grant college degrees. (“Online University...” 1999). 

But how well are these Internet or web-based courses working? Faculty at Florida Gulf 
Coast University expressed some dissatisfaction with web-based courses in a recent survey. They 
complain that web-based courses require extra effort to teach, due to voluminous e-mail 
correspondence with students. They also have reservations about the appropriateness of using the 
Internet to teach courses requiring hands-on training. Questions of copyright also arise in the 
case of syllabi and lectures posted on the Internet. (‘TGCU Faculty. . .” 1998). Despite these 
concerns and others, efforts go forward to evaluate and design the “best” web-based courses 
(Bosher et.al. 1997). 

If web-based instruction is the pedagogical equivalent of classroom instruction we can 

predict: 

Hypothesis: There will be no significant difference in student performance in a web-based 

instructional environment versus a classroom-based instructional environment. 

The study reported herein records a case study comparing web-based and classroom instruction. 

Methodology 

Forty students enrolled in the class. Communication Technology & Change were divided 
into two groups. One group received the course via classroom instruction, the other group 
received the course over the Internet. Because this was a graded course, ethics required that 
students not be randomly assigned to conditions in the experiment. Instead, volunteers were 
requested for the Internet section. Sixteen students (11 males, 5 females) volunteered to 
participate in the Internet section leaving 24 students (14 males, 10 females) in the traditional 
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classroom setting. 

All students enrolled in the web-based section were required to subscribe to a class 
listserve. Written lectures were placed on the Internet. The lectures included the following, 
images and graphics, links to other Internet sites for fiarther reading, discussion questions that 
were linked to the listserve via the “mailto” command. Students in the classroom section did not 
have access to either the listserve or the posted lectures. Students in both sections took exams in 
the classroom at the same time. 

Results 

Because students were not randomly assigned to sections, efforts were made to discover if 
the students in the two sections had similar traits. All students in the class were given the 
following scales prior to the separation of the class into the two sections. A 2-way analysis of 
variance using instructional section and gender as independent variables was performed for each 
scale. No significant interactions were observed between instructional section and gender for any 
of the dependent variables. 

1 . Personal Report of Communication Apprehension (PRCA-24) (Beatty, 1994) In addition 
to allowing students to set their own learning pace, web-based environments give students 
the opportunity to avoid contact with other students. This scale was given to see if 
students opting for the distance learning section might be more apprehensive of interacting 
with other students. The ANOVA tests indicated that the Internet group was significantly 
more apprehensive than the classroom group (see Table 1). 

Verbal Aggressiveness Scale (Rubin, 1994) Persons communicating via Internet chat 
rooms, listserves, Usenet groups and email have a reputation for being more aggressive in 



2 . 



their speech than is usual in face-to-face communication (Dyson 1997). This scale was 
given to see if those opting for the Internet condition were more verbally aggressive than 
those opting to remain in the classroom. No significant differences were found between 
the two groups on this scale (see Table 1). 

3. Computer Apprehension Scale (Gressard and Loyd, 1985) Students opting for the Internet 
section would have to access lectures, ask and answer questions, etc. via computers. It 
seemed Ukely that students with high levels of computer “phobia” would be unlikely to opt 
for web-based instruction. The Computer Apprehension Scale measures Anxiety, 
Confidence and Liking for computers. The ANOVA test on this scale indicated that 
students opting to remain in the classroom had significantly higher levels of computer 
anxiety (see Table 1) and that females were significantly more apprehensive than were 
males (see Table 2). 

4. An Inventory of Computer and Internet Usage (see appendix) As with the previous scale, 
it seemed likely that students opting for the web-based environment would have had more 
experience with computers and the Internet. An inventory of computer and Internet 
activities was given. Students opting for the Internet section had engaged in significantly 
more general computer activities than the classroom section, but there was no significant 
difference in prior Internet usage. (See Table 1) 

If web-based instruction is an adequate substitute for classroom instruction, then student 
performance on graded assignments should not differ. A 2-way Analysis of Variance was 
performed using Instructional section and gender as independent variables and mid-term and final 
exam grades (both tests combined multiple choice and essay questions) and the final grade point 



average for the course (GPA reflected exam grades, web page assignments and Internet 
exercises)as dependent variables. No significant interactions were found between instructional 
condition and gender. Females had significantly higher overall GPA’s than did males but there 
were no significant differences between males and females in mid-term and final exam scores. (See 
Table 2) In all cases the classroom section performed slightly better than the Internet section, but 
in no case did the differences achieve statistical significance. (See Table 1) At least as far as 
graded assignments are concerned, taking the class via the Internet did not seem to affect 
performance either for the better or for the worse. 

The Computer Apprehension Scale was given to students again at the end of the semester 
to see if experience with computers in the course would affect student attitudes toward 
computers. Recall that prior to the beginning of the course, the classroom section was 
significantly more apprehensive of computers. At the end of the semester this difference had 
disappeared and there was no longer any significant difference in computer apprehension between 
the two sections. However, a closer examination of pre- and post-test computer apprehension 
scores indicates that this was achieved by lowering apprehension scores among the classroom 
section and increasing apprehension scores among the Internet section! (see Table 1) 

At the end of the semester, students in the Internet section were asked to fill out a 
questionnaire (see appendbc) asking about their experiences with web-based instruction. No 
significant differences due to gender were discovered. Students who reported accessing the class 
lectures once a week or more often had significantly higher GPA’s and final exam grades. (See 
Table 3) 

While GPA and mid-term exam grades were also significantly related to fi-equency of 
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contact with the instructor via email, no logical pattern emerges. Students seemed to perform 
better if they had either greater than usual contact or less than usual contact with the instructor. 
(See Table 3). 

Students who either printed out all of the lectures or none of the lectures performed better 
overall than those who printed out only some of the lectures (See Table 3). 

Students who reported problems printing out the lectures performed significantly better on 
the mid-term exam than those who reported no problems. (See Table 3) 

Students who reported that their lecture notes were better than they usually take earned 
significantly higher GPA’s than those who reported their lecture notes were the same as usual. 
(See Table 3). 

Conclusions 

Was web-based instruction as effective as classroom instruction? The answer is a 
qualified, “yes.” Yes, as far as all the graded assignments in the course were concerned. No 
significant differences were found between the two groups in regard to mid-term and final exam 
grades or in overall grade point averages for the course. 

Nevertheless, the Internet students did not utilize the listserve adequately. On only three 
occasions throughout the entire 15 week semester did students actually respond to comments 
made by other students on the listserve. All other listserve activity was composed of students 
giving very short answers to questions posed by the instructor that were built in to the lectures or 
responding to questions posed by the instructor that were posted on the hstserve. 

Two problems came to light regarding the use of the listserve. First, students were not 
keeping up with the lectures. One student might respond to a particular lecture’s questions during 
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the week they were supposed to be reading that lecture, while other students would respond to 
the same question two or three weeks later. It is difficult for students to discuss the content of a 
week’s lecture on the listserve if they are not reading the same lecture at the same time. 

Secondly, many students did not use the listserve at all during the semester. Discussions are 
impossible if students refuse to participate. 

From a pedagogical point of view, it is hard to conclude that the students taking the 
course via the Internet received the same quality of instruction as those in the classroom. If 
classroom discussion is important to learning, then they clearly did not. 

Nevertheless, all Internet students who filled out the final questionnaire (two managed to 
avoid it) indicated that they would like to take more classes via the Internet. The general 
impression given by the students was that they really didn’t care about classroom interaction and 
just wanted to earn the credits and get the grades. Only one student in the Internet section 
indicated dissatisfaction with the course. And his complaint was that his fellow students would 
not respond to his listserve postings. (His postings were the only ones that ran longer than a short 
paragraph - he tended to fill pages). He clearly wanted someone to argue with him, and the 
instructor was the only one who would. 

Suggestions for future web-based courses 

The following suggestions are offered based on experience with the case study reported 
herein. Caution should be used in enacting these suggestions as the sample size in the case study 
was very small and students were allowed to self-select which instructional section they wished to 
take. Future research is needed in this area to confirm the findings. 

Regarding Posting Lectures on the Internet 
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Lectures/syUabi should be posed behind a “firewall” to prevent people who are not 
enrolled in the course from accessing these materials. Far too many colleges and 
universities are attempting to make money off web-based instruction these days. 
Professors should not “give away” their work. 

Lectures should include opportunities for students to respond to posted materials. This 
can be achieved by placing discussion questions in the lectures linked via the “mailto” 
command to the listserve. 

Lectures should include appropriate design elements to make them both attractive and 
interesting - but - design cannot take the place of substance. Above all, the content of the 
course should be challenging and thought-provoking. 

Care should be taken to proofi-ead all posted materials for spelhng/grammar, and 
references for lecture material should be posted. 

Avoid the temptation to add audio/video elements to your lectures. Students will be 
accessing lectures through a wide variety of computer hardware, telephone and cable 
modems. Design for the lowest level of equipment. Assume your students are operating 
with an old 486 and a slow telephone modem. Do not assume multi-media capability. 
Regarding Student Class Participation 

Students will probably need to be “encouraged” to participate in listserve discussions. 

You might try counting hstserve activity as a part of a class-participation grade. Be 
prepared to send students a report every other week on how satisfactory their listserve 
performance is. 

In order to “encourage” students to keep up with the rest of the class in their reading and 



participation in listserve discussions, build in regular graded assignments with firm due 
dates. Such assignments could take the form of essays based on reading assignments. 

3. Chat rooms are not recommended. One of the advantages of web-based instruction is that 
students can take the course on their own daily schedule. Trying to get everyone together 
in a chat room at a particular time would be a logistical nightmare. 

Testing 

Students in this case study took tests in the classroom regardless of which instructional 
section they were in. Other arrangements would have to be made for a course offered exclusively 
over the Internet. Such arrangements can typically be made with local colleges/universities, public 
libraries, etc. Or you could design “take-home” style tests. 

A Final Thought 

Don’t let the school administration talk you into letting the enrollment in web-based 
courses climb to astronomical levels. These courses do take more time and effort to teach. You 
can’t “wing” web lectures! All materials must be carefully prepared and designed in advance. 

You will have to answer the same questions over and over again for students who refuse to access 
the listserve on a regular basis. And you will have to build in additional graded assignments to 
encourage students to keep up with their reading and discussion that would not be necessary in a 
classroom setting. Administrators have a nasty habit of basing course enrollment on available 
seating. Imagine their excitement when they realize that a web-based course can have, in theory, 
unlimited enrollment. Take a firm stand on course enrollment right from the start! 
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Table 1 



ANOVAs of Group bv Dependent Variables 



Ilependent Variables 


Groap 


Mean/ Standard 


F-rado 


Significance 


Range 




Deviation 


degrees of freedom 




General Compute Use 
O(low)- 13(hi^) 


In-Class 


7.3333/2.7767 


17.878 


.000* 




Internet 


10.6250/ 1.7078 


1,38 




Internet Activity 
0(low)-20(hi^) 


In-Class 


5.1667/4.4883 


3.357 


.075 




Internet 


7.8750/4.7170 


1,38 




Compute Apprehension 


In-Class 


3.3333/8.7510 


6.418 


.016* 


Prd^t/Posttest Difference 


Internet 


-4.7143/ 10.5643 


1,36 




Computer Apprehension 
Pretest 


In-Class 


72.0000/ 19.9695 


6.142 


.018* 


30(low)-l50(hi^) 


Internet 


57.8750/13.3610 


1,38 




Computer Apprehension 
Posttest 


In-Class 


69.5000/23.6036 


.251 


.620 


30(low)>l50(hi^) 


Internet 


66.2857/4.8267 


1,36 




Grade Point Average for 


In-Class 


3.0383/ .5534 


1.320 


.258 


Class 


Internet 


2.8271/ .5344 


1,36 




Communication 


In-Class 


19.7292/4.7290 


6.150 


.018* 


Apprehension 
6(low) - 30(hi^) 


Internet 


23.3594/4.2219 


1,38 




Mid4enn£xam 


In-Class 


.6767/ .08052 


2.458 


.125 




Internet 


.6250/. 1231 


1,38 




Final Exam 


In-Class 


.8483/.0 8052 


.861 


.359 




Internet 


.8213/. 1039 


1.38 




Verbal Aggression Scale 
20(low) > 100(hi^) 


In-Class 


43.2500/10.9157 


.895 


.350 




Internet 


46.2500/7.8613 


1,38 





* statistically significant at < .05 
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Table 2 



ANOVAs of Sex bv Dependent Variables 



Dependent 


Sex 


Mean/ Standard 


F-ratio 


Significance 


Variables 

Range 




Deviation 


degrees of freedom 




General Computer 
Use 


Male 


8.9231/3.2115 


.659 


.422 


0-13 


Female 


8.1429/2.1788 


1,38 




Internet Activity 


Male 


6.6154/ 4.8256 


.439 


.511 


0-20 


Female 


5.5714/ 4.6029 


1,38 




Computer 

Apprehension 


Male 


-2.4167/ 10.2699 


5.547 


.024* 


Pretest/Posttest 

Difference 


Female 


5.1429/8.1037 


1,36 




Computer 

Apprehension 


Male 


60.4615/ 15.8120 


8.748 


.005* 


Pretest 

30-150 


Female 


77.2857/ 19.4913 


1,38 




Computer 

Apprehension 


Male 


65.2500/ 16.8736 


1.750 


.194 


Posttest 


Female 


73.5714/21.5682 


1,36 




Grade Point 


Male 


2.8117/ .5621 


5.353 


.027* 


Average for Class 


Female 


3.2157/ .4333 


1,36 




Communication 

Apprehension 


Male 


21.5769/5.1627 


.493 


.487 


6-30 


Female 


20.4464/4.2087 


1,38 




Mid-term Exam 


Male 


.6492/ .08993 


.309 


.581 




Female 


.6686/. 1290 


1,38 




Final Exam 


Male 


.8454/ .08453 


.560 


.459 




Female 


.8229/. 1018 


1,38 




Verbal Aggression 
Scale 


Male 


44.3846/7.5423 


.003 


.955 


20-100 


Female 


44.5714/ 13.3918 


1,38 





* statistically significant at < 05 
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Table 3 

Statistically Significant ANOVAs 



Independent Variables 


Dependent Variables 


Mean/ Standard 


F-ratio 


Significance 






Deviation 


degrees of freedom 


Access Lectures 


Grade Point Average 




9.202 


.007* 


twice a week/more 




2.9275/ .4007 


2,9 




once a week 




3.1000/ .0000 






once a month 




1.8000/ .0000 






Access Lectures 


Final Exam 




5.626 


.021* 


twice a week/more 




.8075/ .09896 


2,11 




once a week 




.9600/ .0000 






once a month 




.7300/ .01 155 






Communicate with 


Grade Point Average 




10.524 


.004* 


Instructor via email 






2,9 




same as usual 




2.3600/ .4429 






more than usual 




2.9950/. 1212 






less than usual 




3.5400/ .0000 






Communicate with 


Mid-term Exam 




10.035 


.003* 


Instructor via email 






2,11 




same as usual 




.5100/. 1084 






more than usual 




.6500/ .01155 






less than usual 




.7500/ .08083 






Print Out Lectures 


Grade Point Average 




16.705 


.001* 


yes, all of them 




3.0675/ .3221 


2,9 




yes, some of them 




1.8000/ .0000 






no 




2.5400/ .0000 






Problems Printing 


Mid-term Exam 




6.640 


.028* 


Lectures 






1,10 




yes 




.7400/.0 9238 






no 




.5575/. 1243 






Quality of Lecture Nates 


Grade Point Average 




1,10 


.001* 


same as usual 




1.8000/ .0000 






better than usual 




2.9620/ .3608 







* statistically si^ificant at <.05 
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A. Internet and Computer Use Questionnaire 

B. Distance Learning Response Questionnaire 
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A 

lUS 

lnst,-uctions: This sun-ey is concerned with your experience using computers and the internet. There are no right or 
wrong answers. 

1 . Have vou ever used Windows? 

a. yes 

b. no (skip to question #3 ) 

2. Check off each of the following Windows procedures with which you are famibar: 

^opening files on the hard and floppy drives (usually C and A drives) 

saving files on the hard and floppy drives 

deleting files in File Manager 

^creating fdes/directories in File Manager 

moving/copying flies in File Manager 

installing new software 

writing text in a word processing program 

editing text (cutting/pasting/moving/sizing, etc) in a word processing program 

printing from a word processing program 

creating tables in a word processing program 

creating figures in a word processing program 

fusing SPSS for windows 

fusing an image editor 

3. Do you have a Zip-Net account? 

a. yes 

b. no 

4. Have vou ever surfed the Internet? 

b. ^(this ends the survey for you. Don't worry. By the end of the semester you will have done most of the 
things this survey' asks about!) 
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5. Do you surf the lulemet from a Mac based s>'Stem or a Windows based s> siem? 

a. Mac 

b. Windows 

c. other (specifS' ' 

6. How do YOU access the Internet? check all that apply 

through a lab computer at the University 

from mv home computer via modem to the Universit} 

^from my home computer via modem to a commercial seiv'ice (such as AOL, CompuSen'e, etc.) 

from my home computer via a high speed cable modem (such as Time-Wamer Cable's Roadrunner) 

Other (specify ^ 

7. Which Web Browsers have you used? Check all that apply. 

Netscape Navigator (regular or gold) 

^Microsoft Internet Explorer 

NCSA Mosaic 

^Lynx 

Mac Web/ Win Web 

Internetworks 

Other ^ 



6. Which Internet activities have you participated in? Check all that apply. 

exchanged. e-mail 

attached files to e-mail messages 

^received and read files attached to e-mail messages 




subscribed to listserve groups 
jparticipated actively in listserve discussions 
read Usenet discussions 
subscribed to Usenet groups 
_participated activety in Usenet disw^ussions 
used an internet search engine (such as Yahoo) 
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bookmarked internet addresses for later reference 

created your o>\n page using an HTML editor (such as Netscape Gold or Microsoft Internet Assistant) 

created your o\\n page using HTML Language 

downloaded sofhvare from the internet 

downloaded gif or jpg files from the internet 

purchased products advertised on the internet 

printed out information found ou Web pages 

saved on disc information found on Web pages 
downloaded, unzipped aud read zipped files from the internet 
visited chat-rooms 

actively participated in chat-room discussions 

other 

(sp ecifr' 

) 



7. Do you consider yourself: 

a. an expert Net Surfer 

b. a pretty good Net Surfer 

c. a mediocre Net Surfer 

d. a newbie Net Surfer 



O 
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Distance Learning Resppnse Sheet Name 

Lectures 

1. How often did you access the lectures? 

a. twice a week or more 

b. once a week 

c. several times a month 

d. once a month 

e. less than once a month 

2. Did you print out the lectures? 

a. yes, all of them 

b. yes, most of them 

c. yes, some of them 

d. no (skip to question 4) 

3. Did you experience any problems printing the lectures? 

a. yes 

b. no 

4. Did you dowrdoad/save the lectures on disk or in your personad computer's hard drive? 

a. yes, all of them 

b. yes, most of them 

c. yes, some of them 

d. no 

5. Did you access tlie lectures from on-campus computer labs? 

a. all of the time 

b. most of the time 

c. some of the time 

d. .never ^ 

6. Did you experience any problems using the on-campus computer labs? (check os vnony as apply) 

a. I had problems getting access to a machine (too crowded) 

b. I had problems with computer viruses 

c. I had problems with malfunctioning machines (not due to viruses) 

d. I had problems with the "help desk" people not knowing the answers to my questions 

e. other (specify) 
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7, Conip^e yoijyr e^erience with the on-line lectures to tal^g lecture nptes in closes; Would you say the 
computer lectures were: (choose one response from each group) 

Group 7a 

a. no better/no worse than the lecture notes I usually take in classes 

b. better than the lecture notes I usually take in classes 

c. worse than lecture notes I usually take in classes 

Group 7b 

a. I found the on-line lectures no more or less interesting than in-class lectures usually are 

b. I found the on-line lectures more interesting than in-class lectures usually are 

c. I fbmid tlie on-line lectures less interesting tlian in-class lectures usually are 

8. What did you like most about the on-line lectures? 



9 , What did you like least about the on-line lectures? 
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10. pa you feel tlie Listserve allowed you to participate in class? 

a. I participated through the hstserve about as much as I usually participate in classes 

b. I participated throi^ the listserve more than I usually participate in classes 

c. I participated through the listserve less than I usually participate in classes 

1 i . What did you like most about the Listserve? 



12. What did you like least about the listserve? 



13. How do you feel about using email to communicate with the instructor? 

a. I talked with the instructor via email about as often as I usually talk with my instructors 

b. I talked with the instructor via email more often than I usually talk with my instructors 

c. I talked with the instructor via email less often than I usually talk with my instructors 

14. What did you like most about using email to communicate with the instructor? 



Email 




ERIC 



15. ^Tiat did you like least about using email to communicate with the instructor? 



16, Would you like the opportunity to take more classes via distance learning (using the internet, listserves and email) 

a. yes, I probably would 

b. no, I probably would not 

17. Why/why not? 



best copy AVAILABLE 
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